Image processing system, image processing method, and computer program

ABSTRACT

An image processing system includes: an object detecting unit that detects a moving body object from image data of an image of a predetermined area; an object-occurrence-position detecting unit that detects an occurrence position of the object detected by the object detecting unit; and a valid-object determining unit that determines that the object detected by the object detecting unit is a valid object when the object is present in a mask area set as a non-detection target in the image of the predetermined area and the occurrence position of the object in the mask area detected by the object-occurrence-position detecting unit is outside the mask area.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing system, an imageprocessing method, and a computer program for analyzing an input imageand outputting a monitoring result.

2. Description of the Related Art

There is an image processing system that analyzes an image obtained byphotographing a predetermined area with a monitoring camera and detectsa moving body such as a person or a car to thereby perform monitoring(hereinafter referred to as monitoring system).

FIG. 9 is a diagram of a state of moving body detection in such amonitoring system. A monitoring camera (not shown) photographs, from thefront, a person 51 as a moving body and a sign 52 as a background objectlocated behind the person 51. The person 51 is captured in a moving bodydetection frame 53 as an object to be monitored. In such moving bodydetection processing, it is possible to apply an algorithm forcalculating temporal changes in pixel values in an image and, when adifference among the pixel values is equal to or larger than apredetermined value, determining that a moving body is present in theimage (see JP-A-2006-107457).

As shown in FIG. 9, the person 51 as the object is passing in front ofthe sign 52. No temporal change occurs in the sign 52 because the sign52 is stationary. Only the person 51 is captured in the moving bodydetection frame 53 and detected by the monitoring camera.

In this example, a moving body is not limited to the person 51 and maybe, for example, a car or a bicycle. The background object 52 may be,for example, a house or a tree.

As shown in FIG. 10, when the sunlight is irradiated on a pole 521 ofthe sign 52, the sunlight is reflected on an area 522 on the surface ofthe pole 521 and causes a large luminance change. The same phenomenonoccurs if a background object is an object having a surface that, likethe sign 52, causes reflection of light such as a wall surface of awarehouse made of metal or a curbstone. Although the sign 52 is notmoving, a luminance change in the sign 52 is large. Therefore, suchlight reflection is misdetected as an object via the monitoring camerain the moving body detection algorithm.

FIG. 11 is a diagram of a state in which the person 51 passes asunlight-irradiated side of the sign 52. As shown in FIG. 11, when thesunlight is blocked by the person 51, a shadow 523 is formed in a partof the area 522 on the surface of the pole 521. In this case, theluminance on the surface of the pole 521 substantially changes. Althoughthe person 51 is not passing between the monitoring camera and the sign52, a luminance change is large on the person 51. Therefore, such ashadow is misdetected as an object via the monitoring camera in themoving body detection algorithm.

Such misdetection may occur at night. Normally, as shown in FIG. 12A,light is not emitted from the surface of the pole 521 of the sign 52 atnight. However, as shown in FIG. 12B, when some light such as light of acar is irradiated on the pole 521, the light is reflected on, forexample, an area 524 on the surface of the pole 521 to generatereflected light. In such a case, as in the case explained above,although a moving body such as the person 51 is not passing between themonitoring camera and the sign 52, a luminance change is large.Therefore, the reflected light is misdetected as an object via themonitoring camera in the moving body detection algorithm.

Therefore, in the moving body detection algorithm in the past, forexample, as shown in FIG. 13, processing for setting a mask area 54 in aposition of the pole 521 of the sign 52 as a non-detection area to notperform image processing in the mask area 54 is performed. This makes itpossible to prevent misdetection on the surface of the pole 521.

Another related art includes Japanese Patent No. 3997062.

SUMMARY OF THE INVENTION

However, when such processing for setting a mask area is performed, forexample, as shown in FIG. 14, the person 51 is not detected in the maskarea 54 and is only detected outside the non-detection area as indicatedby the moving body detection frame 53.

Therefore, it is desirable to provide an image processing system, animage processing method, and a computer program that can accuratelydetect a moving body object in an image in which a mask area is set.

According to an embodiment of the present invention, there is providedan image processing system including: an object detecting unit thatdetects a moving body object from image data of an image of apredetermined area; an object-occurrence-position detecting unit thatdetects an occurrence position of the object detected by the objectdetecting unit; and a valid-object determining unit that determines thatthe object detected by the object detecting unit is a valid object whenthe object is present in a mask area set as a non-detection target inthe image of the predetermined area and the occurrence position of theobject in the mask area detected by the object-occurrence-positiondetecting unit is outside the mask area.

According to another embodiment of the present invention, there isprovided an image processing method including the steps of: detecting amoving body object from image data of an image of a predetermined area;detecting an occurrence position of the object detected in the detectingan object; and determining that the object detected in the detecting anobject is a valid object when the object is present in a mask area setas a non-detection target in the image of the predetermined area and theoccurrence position of the object in the mask area detected in thedetecting an occurrence position is outside the mask area.

According to still another embodiment of the present invention, there isprovided a computer program for causing a computer to execute the stepsof: detecting a moving body object from image data of an image of apredetermined area; detecting an occurrence position of the objectdetected in the detecting an object; and determining that the objectdetected in the detecting an object is a valid object when the object ispresent in a mask area set as a non-detection target in the image of thepredetermined area and the occurrence position of the object in the maskarea detected in the detecting an occurrence position is outside themask area.

According to the present invention, even in the outdoors where anenvironmental change tends to occur, when an intruder or the like ismonitored by using a monitoring camera, it is possible to accuratelydetect a moving body object and reduce misreporting of an alert.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a configuration example of a monitoring systemaccording to an embodiment of the present invention;

FIG. 2 is a functional block diagram of a configuration of a monitoringcamera included in the monitoring system according to the embodiment;

FIG. 3 is a diagram of a state of object detection performed when themonitoring system according to the embodiment sets a mask area in abackground object that is a non-detection target;

FIG. 4 is a diagram of a configuration of a metadata generating unit;

FIG. 5 is a flowchart for explaining object detection processingoperation of the monitoring system according to the embodiment;

FIG. 6 is a diagram of a state in which a pan-tilt type monitoringcamera included in the monitoring system according to the embodimentturns;

FIG. 7 is a functional block diagram of a detailed configuration of aclient terminal;

FIG. 8 is a diagram of another configuration example of the monitoringsystem according to the embodiment;

FIG. 9 is a diagram of a state of moving body detection in a monitoringsystem in the past;

FIG. 10 is a diagram of a state in which the sunlight is reflected on anarea in a part on the surface of a pole of a sign, on which the sunlightis irradiated, and causes a large luminance change;

FIG. 11 is a diagram of a state in which a person as an object passes asunlight-irradiated side of a background object;

FIGS. 12A and 12B are diagrams for explaining misdetection of an objectat night;

FIG. 13 is a diagram for explaining mask setting processing in themonitoring system in the past; and

FIG. 14 is a diagram of a state of object detection in the monitoringsystem in the past.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

An embodiment of the present invention is explained in detail below withreference to the accompanying drawings.

FIG. 1 is a diagram of a configuration example of a monitoring systemaccording to an embodiment of the present invention. In a monitoringsystem 100 shown in FIG. 1, a client terminal 3 acquires, via a network,data output from one or plural monitoring cameras.

In the monitoring system 100, three monitoring cameras 1 a, 1 b, and 1 care connected to the client terminal 3 via a network 2. The clientterminal 3 may be any information processing apparatus including adisplay unit such as a personal computer as shown in FIG. 1.

The monitoring cameras 1 a, 1 b, and 1 c respectively photograph, inframe units, a monitoring target moving body in monitoring areas togenerate image data and generate metadata from the image data of eachframe. In this embodiment, examples of a monitoring target objectinclude various moving bodies such as a person, a car, a bicycle, and ananimal.

When the client terminal 3 acquires the image data and the metadata fromthe monitoring cameras 1 a, 1 b, and 1 c via the network 2, the clientterminal 3 displays an image based on the image data on a display unit311. The client terminal 3 causes a storing unit (not shown) to storethe image data and the metadata, analyzes the metadata, and outputs aresult of the analysis.

The metadata acquired by the client terminal 3 from the monitoringcamera 1 a, 1 b, and 1 c via the network 2 is analyzed via a metadatafilter (hereinafter referred to as “filter”). Depending on content of afilter processing result, the client terminal 3 supplies a switchinginstruction signal to the monitoring cameras 1 a, 1 b, and 1 c in orderto control the operation of the monitoring cameras 1 a, 1 b, and 1 cfrom which monitoring images suitable for monitoring are obtained.

Metadata generated in a monitoring camera is explained. The metadata isattribute information of image data of an image picked up by an imagingunit of the monitoring camera. For example, the metadata includes thefollowing:

object information (information such as a position, a motion vector, anID, a coordinate, and size of an object detected by the monitoringcamera);

imaging time data and direction information (pan, tilt, etc.) of themonitoring camera;

position information of the monitoring camera; and

signature information of a picked-up image.

The object information is information obtained by expanding informationdescribed as binary data in the metadata into meaningful data structuresuch as a structure.

The metadata filter is a determination condition in generating alertinformation from object information. The alert information isinformation subjected to filter processing based on the objectinformation expanded from the metadata. The alert information isobtained by analyzing metadata of plural frames, deducing speed from achange in a position of a moving body, checking whether the moving bodycrosses a certain line, or analyzing these kinds of information in acompound manner.

As a type of the filter in the client terminal 3, for example, there areseven types explained below. A filter of an arbitrary type among thesetypes can be selected.

Appearance: a filter for determining whether an object is present in acertain area

Disappearance: a filter for determining whether an object appears in acertain area and moves out of the area

Passing: a filter for determining whether an object crosses a certainboundary

Capacity: a filter for counting the number of objects in a certain areaand determining whether a cumulative number of objects exceeds apredetermined value

Loitering: a filter for determining whether an object stays in a certainarea exceeding a predetermined time

Unattended: a filter for determining whether an object that intrudesinto a certain area and does not move exceeding a predetermined time ispresent

Removed: a filter for detecting that an object present in a certain areais removed

Examples of data included in the alert information include “a cumulativenumber of objects” generated through a filter that uses a cumulativevalue of detected objects such as “Capacity” or the like among thefilter explained above, “the number of objects” as the number of objectsmatching a condition of a filter, the number of objects matching acondition of a filter in a specific frame, and attribute information ofan object matching a condition of a filter (an ID, an X coordinate, a Ycoordinate, and the size of the object). In this way, the alertinformation includes the numbers of objects (the numbers of people) inan image and statistics of the numbers of objects. The alert informationcan also be used as a report function.

In this embodiment, as explained above, the client terminal 3 generatesthe alert information on the basis of the metadata transmitted from themonitoring cameras 1 a, 1 b, and 1 c. In recent years, according toconditions such as a reduction in size and power saving of an arithmeticprocessing device, it is possible to perform, on a camera, processingperformed by a client apparatus in the past. It is also possible togenerate alert information on the basis of a predetermined condition (afilter) in the monitoring cameras 1 a, 1 b, and 1 c.

A detailed configuration of the monitoring cameras 1 a, 1 b, and 1 c isexplained. FIG. 2 is a functional block diagram of a configuration ofthe monitoring camera 1 a. Since the monitoring cameras 1 a, 1 b, and 1c have the same configuration, explanation of configurations of themonitoring cameras 1 b and 1 c is omitted.

The monitoring camera la includes an image-data generating unit 11, animaging-operation switching unit 12, and a metadata generating unit 13.

The image-data generating unit 11 includes a lens unit 111, an imagingunit 112, an imaging-signal processing unit 113, and a data processingunit 114.

The imaging unit 112 performs imaging in frame units through the lensunit 111 and photoelectrically converts imaging light focused on animaging element (not shown) to generate an imaging signal Sv in frameunits.

The imaging unit 112 includes, for example, a preamplifier unit and anA/D (Analog to Digital) converting unit. The preamplifier unit performsamplification of an electric signal level of the imaging signal Sv andremoval of reset noise due to correlated double sampling. The A/Dconverting unit converts the imaging signal Sv from an analog signalinto a digital signal.

The imaging unit 112 also performs gain adjustment, stabilization of ablack level, adjustment of a dynamic range, and the like for thesupplied imaging signal Sv in frame units. Thereafter, the imaging unit112 supplies the imaging signal Sv to the imaging-signal processing unit113.

The imaging-signal processing unit 113 applies various kinds of signalprocessing to the imaging signal Sv supplied from the imaging unit 112and generates image data Dv. The imaging-signal processing unit 113performs, for example, knee correction for compressing a level equal toor higher than a certain level of the imaging signal Sv, γ correctionfor correcting a level of the imaging signal Sv according to a γ curve,and white clip processing an black clip processing for limiting a signallevel of the imaging signal Sv to a predetermined range. Theimaging-signal processing unit 113 supplies the image data Dv to thedata processing unit 114 and the metadata generating unit 13 and causesthe storing unit (not shown) to store the image data Dv.

The data processing unit 114 applies encoding processing to the imagedata Dv in order to reduce a data amount in performing communicationwith the client terminal 3 and the like and generates image data Dt. Thedata processing unit 114 supplies the generated image data Dt to theclient terminal 3 as predetermined data structure. The monitoring system100 performs processing for detecting a moving body object. Therefore,an image including the object is more important than other images notincluding the object. Therefore, in this encoding processing, the dataprocessing unit 114 may increase a compression ratio for the image dataDv of the image including the object and reduce a compression ratio forthe image data Dv of the images not including the object. This makes itpossible to highly precisely reproduce the image including the object.

The imaging-operation switching unit 12 controls the lens unit 111, theimaging unit 112, the imaging-signal processing unit 113, and the dataprocessing unit 114 on the basis of a switching instruction signal CAinput from the client terminal 3 such that an optimum picked-up imagecan be obtained and performs operation switching for the monitoringcamera 1. The imaging-operation switching unit 12 performs, for example,besides performing switching of an imaging direction of the imagingunit, processing for supplying a control signal CMa to the lens unit 111to cause the lens unit 111 to perform switching of a zoom ratio and aniris, supplying a control signal CMb to the imaging unit 112 and theimaging-signal processing unit 113 to cause the imaging unit 112 and theimaging-signal processing unit 113 to perform switching of a frame rateof the picked-up image, and supplying a control signal CMc to the dataprocessing unit 114 to cause the data processing unit 114 to performswitching of a compression ratio of image data.

The imaging-operation switching unit 12 supplies an imaging operationsignal QF (e.g., an imaging direction and a zoom state at the time whenthe monitoring target object is imaged and setting information of theimage-data generating unit 11) to the metadata generating unit 13 andcontrols the metadata generating unit 13.

The metadata generating unit 13 generates metadata Dm includinginformation concerning the object. As in this embodiment, when themoving body is set as the monitoring target object, the metadatagenerating unit 13 detects the moving body object using the image dataDt generated by the image-data generating unit 11, generates objectdetection information indicating whether the object is detected andobject position information indicating a position of the detectedobject, and includes the object detection information and the objectposition information in the metadata Dm as object information. A uniqueID is allocated to the detected object.

Information concerning the monitoring target included in the metadata Dmis not limited to the information related to the object and may beinformation indicating a state of an area monitored by the monitoringcamera, for example, information such as the temperature and thebrightness of the monitored area. Alternatively, the informationconcerning the monitoring target may be information concerning, forexample, operation performed in the monitored area. When the temperatureis set as a monitoring target, the metadata generating unit 13 only hasto include a temperature measurement result in the metadata Dm. When thebrightness is set as a monitoring target, the metadata generating unit13 only has to discriminate, for example, average luminance of amonitoring image on the basis of the image data Dv and include a resultof the discrimination in the metadata Dm.

When operation performed by a user on an ATM (Automated Teller Machine),a POS (Point Of Sales), and the like is set as a monitoring target, themetadata generating unit 13 only has to include user operation performedthrough an operation key, an operation panel, and the like in themetadata Dm.

The metadata generating unit 13 can record and keep time and a situationof generation of the metadata Dm by including the imaging operationsignal QF supplied from the imaging-operation switching unit 12 (e.g.,an imaging direction and a zoom state at the time when the monitoringtarget is imaged and setting information of the image-data generatingunit 11), time information, and the like in the metadata Dm.

The structure of the image data Dt and the metadata Dm is explained. Theimage data Dt and the metadata Dm include data main bodies and linkinformation. The data body of the image data Dt is image data ofmonitoring images photographed by the monitoring cameras 1 a and 1 b.The data body of the metadata Dm is the description of information andthe like indicating the monitoring target object and attributeinformation defining a description mode for the information. On theother hand, the link information is the description of associationinformation indicating association between the image data Dt and themetadata Dm, attribute information defining a description mode ofcontent of the information, and the like.

As the association information, for example, a time stamp and a sequencenumber for specifying the image data Dt are used. The time stamp isinformation indicating generation time of the image data Dt (timeinformation). The sequence number is information indicating generationorder of content data (order information). When plural monitoring imageshaving the same time stamp are present, generation order of the imagedata Dt having the same time stamp can be identified. As the associationinformation, information for specifying an apparatus that generates theimage data Dt (e.g., a manufacturing company name, a model name, and aserial number) may be used.

For the description of the link information and the metadata body, amarkup language defined for describing information exchanged on the web(WWW: World Wide Web) is used. When the markup language is used,exchange of information via the network 2 can be easily performed.Further, exchange of image data and metadata can also be easilyperformed by using, as the markup language, for example, the XML(Extensible Markup Language) used for exchange of documents andelectronic data. When the XML is used, for example, an XML schemer isused as the attribute information defining the description mode of theinformation.

The image data Dt and the metadata Dm generated by the monitoringcameras 1 a, 1 b, and 1 c may be supplied to the client terminal 3 asone stream. Alternatively, the image data Dt and the metadata Dm may besupplied to the client terminal 3 asynchronously as separate streams.

In the object detection processing in the past, in order to preventmisdetection in an area of a non-detection target, a mask area is setfor the non-detection target and image processing is not performed inthe mask area. However, in such object detection processing in the past,a moving body is not detected when the moving body is present in themask area.

Therefore, the monitoring system 100 according to this embodimentperforms object detection processing for detecting an object even whenthe object intrudes into the mask area.

In this object detection processing, for example, as shown in FIG. 3,when a mask area 64 as a non-detection area is set in a position of apole 621 of a sign 62, a person 61 as an object is detected in the maskarea 64 even when the person 61 intrudes into the mask area 64 from theoutside of the mask area 64. The person 61 is captured in a moving bodydetection frame 63 and detected.

FIG. 4 is a diagram of a configuration of the metadata generating unit13. The metadata generating unit 13 includes an object detecting unit131, an object-occurrence-position detecting unit 132, a valid-objectdetermining unit 133, and a valid-metadata generating unit 134 andperforms such moving body detection processing.

The metadata generating unit 13 generates the metadata Dm includinginformation concerning an object. The metadata generating unit 13detects the moving body object using the image data Dt generated by theimage-data generating unit 11, generates object detection informationindicating whether the object is detected and occurrence positioninformation indicating an occurrence position of the detected object,and includes the object detection information and the occurrenceposition information in the metadata Dm as object information. Thedetected object is allocated with a unique ID and stored in the storingunit (not shown).

In the monitoring system 100, before the metadata generating unit 13performs the moving body detection processing, the client terminal 3sets, on the basis of operation of the user, a mask as a non-detectionarea in a place that is likely to be excluded from object detectionprocessing. A mask setting signal as mask setting information issupplied from the client terminal 3 to the monitoring cameras 1 a, 1 b,and 1 c via the network 2.

The image data Dv of the moving body object such as the person 61 issupplied to the object detecting unit 131 from the imaging-signalprocessing unit 113. In the imaging system 100, the imaging unit 112photographs an image for each frame. The imaging-signal processing unit113 applies image processing to the imaging signal Sv in frame unitsacquired by the imaging unit 112. The image data Dv for each frame issupplied to the object detecting unit 131. The object detecting unit 131subjects the image data Dv supplied from the imaging-signal processingunit 113 to image processing and acquires the object. The objectdetecting unit 131 sets, as the metadata Dm, object detectioninformation indicating whether the object is detected. The objectdetecting unit 131 allocates a unique ID to the detected object. Theobject detecting unit 131 includes the object detection information inthe metadata Dm as object information and causes the storing unit (notshown) to store the metadata Dm. The object detecting unit 131 transmitsthe metadata Dm to the client terminal 3.

In the object detection processing performed by the imaging-signalprocessing unit 113, for example, the moving body detection algorithmdisclosed in JP-A-2006-107457 can be applied. In some case, pluralimages of an object are acquired. In this case, the same algorithm isapplied. In the algorithm disclosed in JP-A-2006-107457, a temporalluminance change is detected in an image in frame units and an area inwhich a change in luminance occurs is set as a moving body object.However, the obtained object also includes a misdetected invalid object.A valid object explained later is an object that is a monitoring targetand for which an alert needs to be reported if the object meets acondition of a filter when the object is detected. The invalid object isan object that is not a monitoring target and for which an alert doesnot need to be reported even if the object is detected. Such an invalidobject is removed by processing explained later performed by thevalid-object determining unit 133.

The monitoring camera 1 a performs processing for tracking the objectdetected on the basis of the luminance change after the occurrence ofthe object. This makes it possible to specify a relation between themask area and an object occurrence position. More specifically, theobject-occurrence-position detecting unit 132 specifies, from the imagedata Dv of all the frames imaged and stored in the storing unit (notshown), an image frame at a point when the object occurs. Theobject-occurrence-position detecting unit 132 detects an occurrenceposition of the object from the frame image and generates objectoccurrence position information. The object-occurrence-positiondetecting unit 132 includes the object occurrence position informationin the metadata Dm as object information and causes the storing unit(not shown) to store the metadata Dm. The object-occurrence-positiondetecting unit 132 transmits the metadata Dm to the client terminal 3.

As a method of detecting an object occurrence position adopted by theobject-occurrence-position detecting unit 132, for example, an objectarea tracking method disclosed in JP-A-2007-334631 can be applied. Withthe object area tracking method, even when an object area temporarilydisappears because of overlap or hiding, identification informationassociated with the object area can be maintained and object trackingcan be performed with high performance.

When the object is detected because of the luminance change in the maskarea, the object-occurrence-position detecting unit 132 may performprocessing for calculating a motion vector for an image around theobject and calculating a position where the object is present in animage of an immediately preceding frame. In this case, the valid-objectdetermining unit 133 explained later determines, on the basis of themotion vector obtained by the vector calculation processing, that theobject present outside the mask area in the image of the immediatelypreceding frame does not occur in the mask area and is a valid object.When a motion vector is not calculated regardless of the fact that thereis a luminance change in the image in the mask area, the valid-objectdetermining unit 133 determines that the detected object is reflectedlight or a shadow and misdetection occurs.

It is possible to further improve accuracy of object determination byperforming these kinds of processing in combination.

The valid-object determining unit 133 determines whether an objectpresent in the mask area among all objects (including an invalid object)already detected by the object detecting unit 131 so far in image dataof all the frames occurs in the mask area or occurs outside the maskarea. In other words, the valid-object determining unit 133 determineswhether an occurrence position of an object present in the mask areadetected by the object-occurrence-position detecting unit 132 is insidethe mask area.

When the object occurs outside the mask area, the valid-objectdetermining unit 133 determines the object as a valid object. When theobject occurs in the mask area, the valid-object determining unit 133determines the object as an invalid object.

With such processing, the valid-object determining unit 133 candetermine whether an object intrudes into the mask area from the outsideof the mask area or the object occurs in the mask area. The valid-objectdetermining unit 133 determines that an object that moves only in themask area is not a valid object. In this determination processing, it isalso possible to further set a determination criterion that even anobject that occurs in the mask area is a valid object if the objectmoves out from the mask area.

The valid-metadata generating unit 134 generates the metadata Dm of themoving body that is information indicating that the object is determinedas the valid object by the valid-object determining unit 133. Themetadata Dm is supplied to the client terminal 3 via the network 2. Theclient terminal 3 reports this alert according to filter processing. Themonitoring camera itself that detects the object may report this alerton the basis of the metadata Dm.

Consequently, when the monitoring system 100 monitors, for example, anintruder, it is possible to reduce misreporting from that in the pastand report an alert indicating that the moving body object is detected.

As determination criteria for a valid object adopted by the valid-objectdetermining unit 133, it is also possible to further add a determinationcriteria based on not only presence or absence of a valid objectexplained above but also motions of an object, for example, whether theobject passes a certain imaginary line or whether the object intrudesinto an imaginary area.

In the object detection processing in the past, object detection isperformed according to only information concerning a luminance change ofan image. Therefore, a background object that causes a large luminancechange such as reflected light or a shadow is often misdetected as amoving body object. On the other hand, with the monitoring system 100that performs the object detection processing explained above, adetected object is accurately detected as a moving body by detecting anoccurrence position of the object.

Object detection processing operation in the monitoring system 100 isexplained with reference to a flowchart in FIG. 5. In the followingexplanation, the monitoring camera 1 a detects an object. However, sincethe monitoring cameras 1 a, 1 b, and 1 c have the same configuration,object detection processing is performed in the same processing stepswhen the monitoring camera 1 b or 1 c detects an object.

In step S1, the monitoring system 100 starts the object detectionprocessing operation.

In step S2, the client terminal 3 performs mask setting processing onthe basis of operation of the user and supplies a mask setting signal tothe imaging-operation switching unit 12 of the monitoring camera 1 a.

In step S3, the imaging unit 112 of the monitoring camera 1 aphotographs a monitoring area in frame units such as 30 fps or 15 fpsvia the lens unit 111 on the basis of control operation of theimaging-operation switching unit 12. The imaging unit 112 supplies theimaging signal Sv in frame units to the imaging-signal processing unit113. The imaging unit 112 supplies the imaging signal Sv to theimaging-signal processing unit 113. The imaging-signal processing unit113 applies the various kinds of signal processing explained above tothe imaging signal Sv to generate the image data Dv and supplies theimage data Dv to the metadata generating unit 13.

In step S3, the imaging-operation switching unit 12 performs masksetting processing for a non-detection area based on the mask settingsignal set in step S2. The imaging-operation setting unit 12 suppliesdata of a set mask (mask data) to the valid-object determining unit 133included in the metadata generating unit 13.

The mask setting processing is not limited to the setting of a maskbased on operation of the user. The client terminal 3 or the monitoringcamera 1 a may statistically calculate an area in which misdetectionfrequently occurs and set the area as a mask area.

In step S4, when the image data Dv is supplied from the imaging-signalprocessing unit 113, the object detecting unit 131 included in themetadata generating unit 13 of the monitoring camera 1 a performsprocessing for detecting a moving body object in this image.

In step S4, the object detecting unit 131 determines whether there is anobject. When the object detecting unit 131 determines in step S4 thatthere is an object, the processing proceeds to step S5. When the objectdetecting unit 131 determines that there is no object, the processingreturns to step S3.

In step S5, the valid-object determining unit 133 determines whether aposition of the object detected by the object detecting unit 131 isinside the mask area. When the position of the object is inside the maskarea in step S5, the processing proceeds to step S6. When the positionof the object is outside the mask area, the processing proceeds to stepS7.

In step S6, the object-occurrence-position detecting unit 132 specifiesan image frame at a point when the object occurs from the image data Dvof all frames imaged and stored in the storing unit (not shown) anddetects an occurrence position of the object in the image. Thevalid-object determining unit 133 determines whether the occurrenceposition of the object detected by the object-occurrence-positiondetecting unit 132 is inside the mask area. When the occurrence positionof the object is outside the mask area in step S6, i.e., when the objectoccurs outside the non-detection area set as the mask, the processingproceeds to step S7. When the occurrence position of the object isinside the mask area, the processing returns to step S3.

In step S7, the valid-metadata generating unit 134 generates themetadata Dm that is information indicating that the moving body objectis detected.

In step S8, the client terminal 3 subjects the generated metadata Dm tofilter processing to generate alert information. The client terminal 3performs alert reporting on the basis of the alert information.Alternatively, such filter processing and generation of alertinformation can also be performed in the monitoring camera 1 a.Thereafter, the processing returns to step S3.

As explained above, in the monitoring system 100, if an occurrenceposition of an object is outside the mask area, the object is determinedas a valid object.

In the processing steps shown in the flowchart of FIG. 5, while anobject that occurs in the mask area is present in the present mask area,the object is determined as an invalid object. Thereafter, tracking ofthe object is continued. When the object moves out from the mask areawhile the tracking is performed, the object is determined as a validobject at this point. In this case, at the point when the object isdetermined as a valid object, it is also possible to trace back imagedata and recognize that the object is valid from the time of occurrenceof the object.

Although not shown in the flowchart of FIG. 5, in the monitoring system100, even if an occurrence position of an object is inside the maskarea, if a present position of the object is outside the mask area, theobject is determined as a valid object.

The configuration of the monitoring cameras 1 a, 1 b, and 1 c in themonitoring system 100 according to this embodiment is not limited to thestationary camera explained above and may be, for example, a pan-tilttype camera having a turnable mechanism.

In the stationary camera, an imaging range is typically fixed if onceset. In the case of such a stationary camera, a mask is typically set ina fixed area.

On the other hand, when a monitoring camera is the pan-tilt type camera,the monitoring camera can perform imaging in a wide range by turning.Consequently, the user can set a mask in a panoramic image generated bythe imaging in the wide range.

An operation example of the pan-tilt type monitoring camera is explainedbelow.

As shown in FIG. 6, a pan-tilt type monitoring camera 7 turns tophotograph an area surrounded by an imaginary cylindrical surface 70indicated by an arc. The monitoring camera 7 turns to photograph, forexample, images having fields of view indicated by an imaging field ofview A, an imaging field of view B, and an imaging field of view C. Themonitoring camera 7 causes the storing unit (not shown) to store valuesof pan and tilt of the monitoring camera 7. The monitoring camera 7projects, on the basis of the pan and tilt values stored in the storingunit, the photographed images on the imaginary cylindrical surface 70having a center same as a rotation center of the camera to therebycreate one panoramic image on the imaginary cylindrical surface 70. Byusing the created image, a mask can be set in the same manner as thecase in which the monitoring camera is the stationary camera. Althoughthe images are projected on the imaginary cylindrical surface 70, it isalso possible to project the images on a spherical surface or projectthe images on an expanded plane such as the Mercator map.

When the monitoring camera 7 may be unable to store a tilt value, it isalso possible to extract feature values of images photographed in anarea A, an area B, and an area C with, for example, edge extraction orcorner extraction to perform matching processing for feature points andcreate one panoramic image. This is effective, for example, when therotation center of the monitoring camera 7 does not coincide with afocus of an image.

In this embodiment, it is also possible to use a method of, for example,extracting, in a slit shape, a part of images photographed by rotatingthe monitoring camera 7 and rearranging the extracted images accordingto the rotation of the monitoring camera 7.

In this embodiment, a zoom-type monitoring camera can be used. In thezoom-type monitoring camera, for example, it is possible to set a maskon the basis of images photographed at wide ends via a lens.

A detailed configuration of the client terminal 3 shown in FIG. 1 isexplained with reference to a functional block diagram of FIG. 7.Respective functional blocks of the client terminal 3 may be configuredby hardware or may be configured by software.

The client terminal 3 includes a network connecting unit 301 thatperforms data transmission with the monitoring cameras 1 a, 1 b, and 1c, an image buffer unit 302 that acquires image data from the monitoringcameras 1 a, 1 b, and 1 c, a metadata buffer unit 303 that acquiresmetadata from the monitoring cameras 1 a, 1 b, and 1 c, a filter settingdatabase (DB) 307 that accumulates filter setting corresponding tofilter processing, a metadata filter unit 306 as a filter unit thatperforms filter processing for the metadata, a rule switching unit 308that notifies the monitoring cameras 1 a, 1 b, and 1 c of a settingchange, an image data accumulation database 304 that accumulates theimage data, a metadata accumulation database 305 that accumulates themetadata, a display unit 311 that displays the image data, the metadata,and the like, an image-data processing unit 309 that performs processingfor causing the display unit 311 to reproduce the image data, a metadataprocessing unit 310 that performs processing for causing the displayunit 311 to reproduce the metadata, and a reproduction synchronizingunit 312 that synchronizes reproduction of the metadata and reproductionof the image data.

The image buffer unit 302 acquires the image data Dt from the monitoringcameras 1 a, 1 b, and 1 c and performs decoding processing for theencoded image data Dt. The image buffer unit 302 stores the acquiredimage data Dt in a not-shown buffer provided in the image buffer unit302. The image buffer unit 302 also performs processing for sequentiallysupplying the image data stored in the not-shown buffer to the displayunit 311 that displays an image. By storing the image data in thenot-shown buffer in this way, it is possible to sequentially supply theimage data to the display unit 311 regardless of reception timing of theimage data transmitted from the monitoring cameras 1 a, 1 b, and 1 c.The image buffer unit 302 causes, on the basis of a recording requestsignal supplied from the rule switching unit 308 explained later, theimaged data accumulation database 304 to accumulate the stored imagedata. It is also possible to cause the image data accumulation database304 to accumulate encoded image data and decode the image data in theimage-data processing unit 309 explained later.

The metadata buffer unit 303 stores the metadata Dm acquired from themonitoring cameras 1 a, 1 b, and 1 c in a not-shown buffer provided inthe meta data buffer unit 303. The metadata buffer unit 303 alsoperforms processing for supplying the stored metadata Dm to the metadatafilter unit 306 explained later. Since the not-shown buffer stores themetadata, it is possible to sequentially supply the metadata to thedisplay unit 311 regardless of reception timing of the metadata Dm fromthe monitoring cameras 1 a, 1 b, and 1 c.

When the metadata buffer unit 303 accumulates the metadata Dm acquiredfrom the monitoring cameras 1 a, 1 b, and 1 c in the metadataaccumulation database 305, the metadata buffer unit 303 adds timeinformation of image data that synchronizes with the metadata. Thismakes it possible to read out the metadata Dm at desired time from themetadata accumulation database 305 using the added time informationwithout discriminating time when content of the metadata Dm is read out.

The filter setting database 307 accumulates filter setting correspondingto filter processing performed in the metadata filter unit 306 explainedlater and supplies the filter setting to the metadata filter unit 306.The filter setting is setting for indicating, for each kind ofinformation concerning a monitoring target included in metadata, forexample, determination criteria for determining whether it is necessaryto perform output of alert information or the like and switching of theimaging operation of the monitoring cameras 1 a, 1 b, and 1 c. Byperforming filter processing for the metadata using the filter setting,it is possible to indicate a filter processing result for each kind ofinformation concerning an object. Depending on the filter processingresult, it may be necessary to perform output of alert information or itmay be necessary to switch the imaging operation of the monitoringcameras 1 a, 1 b, and 1 c.

The metadata filter unit 306 performs the filter processing for themetadata using the filter setting accumulated in the filter settingdatabase 307 and determines whether an alert should be reported. Themetadata filter unit 306 performs the filter processing for the metadataacquired by the metadata buffer unit 303 and the metadata supplied fromthe metadata accumulation database 305 and notifies the rule switchingunit 308 of a result of the filter processing.

The rule switching unit 308 generates a switching instruction signal onthe basis of the filter processing result notified from the metadatafilter unit 306 and notifies the monitoring cameras 1 a, 1 b, and 1 c ofa change such as switching of an imaging direction. For example, therule switching unit 308 outputs, on the basis of the filter processingresult obtained from the metadata filter unit 306, a command forswitching the operation of the monitoring cameras 1 a, 1 b, and 1 c suchthat a monitoring image suitable for monitoring can be obtained. Therule switching unit 308 supplies a recording request signal to the imagedata accumulation database 304 on the basis of the filter processingresult and causes the image data accumulation database 304 to accumulatethe image data acquired by the image buffer unit 302.

The image data accumulation database 304 accumulates the image dataacquired by the image buffer unit 302. The metadata accumulationdatabase 305 accumulates the metadata Dm acquired by the metadata bufferunit 303.

The image-data processing unit 309 performs processing for causing thedisplay unit 311 to display the image data accumulated in the image dataaccumulation database 304. The image-data processing unit 309sequentially reads out the image data from a reproduction positionindicated by the user and supplies the read-out image data to thedisplay unit 311. The image-data processing unit 309 supplies areproduction position (reproduction time) of image data being reproducedto the reproduction synchronizing unit 312.

The reproduction synchronizing unit 312 that synchronizes the metadataDm and the image data supplies a synchronization control signal to themetadata processing unit 310 and controls the operation of the metadataprocessing unit 310 such that the reproduction position supplied fromthe image-data processing unit 309 and a reproduction position at thetime when the metadata processing unit 310 reproduces the metadataaccumulated in the metadata accumulation database 305 synchronize witheach other.

The metadata processing unit 310 performs processing for causing thedisplay unit 311 to display the metadata Dm accumulated in the metadataaccumulation database 305. The metadata processing unit 310 sequentiallyreads out the metadata Dm from the reproduction position indicated bythe user and supplies the read-out metadata Dm to the display unit 311.When both the image data and the metadata Dm are reproduced, asexplained above, the metadata processing unit 310 controls thereproduction operation on the basis of the synchronization controlsignal supplied from the reproduction synchronizing unit 312 and outputsthe metadata Dm that synchronizes with the image data to the displayunit 311.

The display unit 311 displays the live image data supplied from theimage buffer unit 302, the reproduced image data supplied from theimage-data processing unit 309, the live metadata Dm supplied from themetadata buffer unit 303, and the reproduced metadata supplied from themetadata processing unit 310. The display unit 311 displays (outputs),on the basis of the filter setting from the metadata filter unit 306, animage indicating a monitoring result based on the filter processingresult using any one of a monitoring image, an image of the metadata,and an image of the filter setting or an image formed by combining theseimages.

The display unit 311 functions as a graphical user interface (GUI). Theuser can define a filter and GUI-display, for example, an analysisresult of information concerning the processing units and alertinformation by selecting a filter setting menu and the like displayed onthe display unit 311 using a not-shown operation key, mouse, remotecontroller, or the like.

With the monitoring system 100 according to this embodiment, it ispossible to accurately detect a moving body object in the outdoors wherean environmental change tends to occur.

In the outdoors, reflected light and a shadow are caused in backgroundobjects such as a sign, a signboard, and a curbstone by the sunlight,light of a car, and the like all day and all night. In the moving bodydetection processing in the past, a stationary mask is mainly set. Theuser sets a large number of stationary masks in the background objectsin order to prevent misdetection of an object. Therefore, a moving bodyobject that should originally be detected may not be able to be detectedand misreporting of an alert, which is a serious problem in a monitoringsystem, often occurs. On the other hand, with the monitoring system 100according to this embodiment, it is possible to reduce the misreportingof an alert by performing the detection processing for an objectoccurrence position explained above.

The present invention is not limited to only the embodiment explainedabove. It goes without saying that various modifications are possiblewithout departing from the spirit of the present invention.

For example, as shown in FIG. 8, the monitoring system according to thisembodiment may further include a server 4 in the configuration of themonitoring system 1 to divide a client function. In other words, amonitoring system 200 according to this embodiment includes the server 4as shown in FIG. 8 and can obtain functions and effects same as those inthe example shown in FIG. 1. In the monitoring system 200 shown in FIG.8, the server 4 has a configuration same as that of the client terminal3, acquires, via the network 2, data output from the monitoring cameras1 a, 1 b, and 1 c, and supplies the data to the client terminal 3. Withsuch a monitoring system 200, by dividing a server function and a clientfunction, it is possible to properly use the server 4 and the clientterminal 3, for example, to process a large amount of data in the server4 having high processing performance and solely view a processing resultin the client terminal 3 having low processing performance. In this way,the monitoring system 200 can be configured as a monitoring system moreabundant in flexibility by distributing the functions via the server.

It goes without saying that the numbers and configurations of monitoringcameras, client terminals, and servers included in the monitoring systemaccording to this embodiment are not limited to the examples explainedabove.

In the explanation of the embodiment, the monitoring cameras 1 a, 1 b,and 1 c transmits the metadata Dm to the client terminal 3 or the server4 and the client terminal 3 or the server 4 generates the alertinformation on the basis of the metadata Dm. However, the presentinvention is not limited to this. The monitoring cameras 1 a, 1 b, and 1c may include metadata processing units and generate and output alertinformation.

The present application contains subject matter related to thatdisclosed in Japanese Priority Patent Application JP 2008-164856 filedin Japan Patent Office on Jun. 24, 2008, the entire contents of which ishereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An image processing system including a computer having one or moremicroprocessors, comprising: an object detecting unit, implemented bythe one or more microprocessors, configured to detect a moving bodyobject from image data of an image of a predetermined area; anobject-occurrence-position detecting unit, implemented by the one ormore microprocessors, configured to detect an occurrence position of themoving body object detected by the object detecting unit, the occurrenceposition of the moving body object corresponding to a position at whichthe moving body object initially appears in the image data; and avalid-object determining unit, implemented by the one or moremicroprocessors, configured to determine that the moving body objectdetected by the object detecting unit is a valid object, when the movingbody object is present in a mask area set as a non-detection target inthe image of the predetermined area and the occurrence position of themoving body object in the mask area, detected by theobject-occurrence-position detecting unit, is outside the mask area. 2.The image processing system according to claim 1, further comprising: avalid-metadata generating unit, implemented by the one or moremicroprocessors, configured to generate, when the valid-objectdetermining unit determines that the object is the valid object,metadata indicating that the moving body object is detected.
 3. Theimage processing system according to claim 1, further comprising: animaging device configured to photograph the predetermined area andgenerates an imaging signal; and an imaging-signal processing unit,implemented by the one or more microprocessors, configured to applypredetermined image processing to the imaging signal generated by theimaging unit and generate image data of the image of the predeterminedarea.
 4. The image processing system according to claim 3, wherein theimaging device generates the imaging signal for each frame.
 5. An imageprocessing method implemented by a computer having one or moremicroprocessors, comprising the steps of: detecting, using the one ormore microprocessors, a moving body object from image data of an imageof a predetermined area; detecting, using the one or moremicroprocessors, an occurrence position of the moving body objectdetected in the detecting an object, the occurrence position of themoving body object corresponding to a position at which the moving bodyobject initially appears in the image data; and determining, using theone or more microprocessors, that the moving body object detected in thedetecting an object is a valid object when the moving body object ispresent in a mask area set as a non-detection target in the image of thepredetermined area and the occurrence position of the moving body objectin the mask area detected in the detecting an occurrence position, isoutside the mask area.
 6. A non-transitory computer readable mediumhaving stored thereon a computer program that when executed by acomputer causes the computer to execute the steps of: detecting a movingbody object from image data of an image of a predetermined area;detecting an occurrence position of the moving body object detected inthe detecting an object, the occurrence position of the moving bodyobject corresponding to a position at which the moving body objectinitially appears in the image data; and determining that the movingbody object detected in the detecting an object is a valid object whenthe moving body object is present in a mask area set as a non-detectiontarget in the image of the predetermined area and the occurrenceposition of the moving body object in the mask area detected in thedetecting an occurrence position is outside the mask area.